Collaborative filtering¶
A distinction is often made between two forms of data collection for recommendation systems. Explicit feedback relies on the user giving explicit signals about their preferences i.e. review ratings. Where as, implicit feedback refers to non-explicit signals of preference e.g. user watch-time. Traditionally, recommender systems can be split into three types:
Collaborative filtering (CF): CF produces recommendations based on the knowledge of users’ attitudes towards items, that is, it uses the “wisdom of the crowd” to recommend items.
Content-based (CB): CB recommender systems focus on the attributes of the items to recommend other items similar to what the user likes, based on their previous actions or explicit feedback.
Hybrid recommendation systems: Hybrid methods are a combination of CB recommending and CF methods
In many applications, content-based features are not easy to extract, and thus, collaborative filtering approaches are preferred. Thus, we will only explore collaborative filtering methods from now on.
CF methods typically fall into three types, memory-based, model-based and more recently deep-learning based (Su & Khoshgoftaar, 2009, He et al., 2017). Neighbour-based CF and item-based/user-based top-N recommendations are typical examples of memory-based systems that utilises user rating data to compute the similarity between users or items. As mentioned previously, common model-based approaches include Bayesian networks, latent semantic models and markov decision processes. In this investigation, we will utilise a weighted matrix factorization approach. Later on, we will generalize the matrix factorization algorithm via a non-linear neural architecture (a softmax model).
However, there are a number of limitations to our approaches such as the inability to model the order of interactions. For instance, Markov chain algorithms (Rendle et al., 2010) can not only encode the same information as traditional CF methods but also the order in which user’s interacted with the items. Furthermore, the sparsity of the frequency matrix (described later on), makes computations prohibitly expensive in real-world settings, without some optimization.
Quick Links:¶
Setup¶
The next few code cells details the initial preparatory steps needed for the development of our collaborative filtering models, namely importing the required libraries; scaling the ids of users and artists;constructing a indicator variable for presence of user-artist interaction;finding the most assigned tag of an artist.
from __future__ import print_function
import numpy as np
import pandas as pd
import collections
from IPython import display
from matplotlib import pyplot as plt
import sklearn
import sklearn.manifold
import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
tf.logging.set_verbosity(tf.logging.ERROR)
# Add some convenience functions to Pandas DataFrame.
pd.options.display.max_rows = 10
pd.options.display.float_format = '{:.3f}'.format
# Install Altair and activate its colab renderer.
print("Installing Altair...")
!pip install git+git://github.com/altair-viz/altair.git
import altair as alt
alt.data_transformers.enable('default', max_rows=None)
alt.renderers.enable('colab')
print("Done installing Altair.")
2021-11-28 22:47:25.531632: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-11-28 22:47:25.531669: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
WARNING:tensorflow:From /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/tensorflow/python/compat/v2_compat.py:111: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
Installing Altair...
Collecting git+git://github.com/altair-viz/altair.git
Cloning git://github.com/altair-viz/altair.git to /tmp/pip-req-build-td6lqo30
Running command git clone --filter=blob:none -q git://github.com/altair-viz/altair.git /tmp/pip-req-build-td6lqo30
Resolved git://github.com/altair-viz/altair.git to commit a987d04e276106f62d4247ea48a1fcead2d06636
Installing build dependencies ... ?25l-
\
|
/
done
?25h Getting requirements to build wheel ... ?25l-
done
?25h Preparing metadata (pyproject.toml) ... ?25l-
done
?25hRequirement already satisfied: entrypoints in /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages (from altair==4.2.0.dev0) (0.3)
Requirement already satisfied: pandas>=0.18 in /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages (from altair==4.2.0.dev0) (1.3.4)
Requirement already satisfied: jinja2 in /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages (from altair==4.2.0.dev0) (3.0.3)
Requirement already satisfied: numpy in /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages (from altair==4.2.0.dev0) (1.21.4)
Collecting toolz
Downloading toolz-0.11.2-py3-none-any.whl (55 kB)
?25l
|█████▉ | 10 kB 28.6 MB/s eta 0:00:01
|███████████▊ | 20 kB 33.1 MB/s eta 0:00:01
|█████████████████▋ | 30 kB 13.6 MB/s eta 0:00:01
|███████████████████████▌ | 40 kB 6.2 MB/s eta 0:00:01
|█████████████████████████████▍ | 51 kB 7.3 MB/s eta 0:00:01
|████████████████████████████████| 55 kB 4.6 MB/s
?25hRequirement already satisfied: jsonschema<4.0,>=3.0 in /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages (from altair==4.2.0.dev0) (3.2.0)
Requirement already satisfied: six>=1.11.0 in /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages (from jsonschema<4.0,>=3.0->altair==4.2.0.dev0) (1.16.0)
Requirement already satisfied: importlib-metadata in /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages (from jsonschema<4.0,>=3.0->altair==4.2.0.dev0) (4.8.2)
Requirement already satisfied: pyrsistent>=0.14.0 in /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages (from jsonschema<4.0,>=3.0->altair==4.2.0.dev0) (0.18.0)
Requirement already satisfied: setuptools in /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages (from jsonschema<4.0,>=3.0->altair==4.2.0.dev0) (47.1.0)
Requirement already satisfied: attrs>=17.4.0 in /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages (from jsonschema<4.0,>=3.0->altair==4.2.0.dev0) (21.2.0)
Requirement already satisfied: pytz>=2017.3 in /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages (from pandas>=0.18->altair==4.2.0.dev0) (2021.3)
Requirement already satisfied: python-dateutil>=2.7.3 in /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages (from pandas>=0.18->altair==4.2.0.dev0) (2.8.2)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages (from jinja2->altair==4.2.0.dev0) (2.0.1)
Requirement already satisfied: zipp>=0.5 in /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages (from importlib-metadata->jsonschema<4.0,>=3.0->altair==4.2.0.dev0) (3.6.0)
Requirement already satisfied: typing-extensions>=3.6.4 in /opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages (from importlib-metadata->jsonschema<4.0,>=3.0->altair==4.2.0.dev0) (4.0.0)
Building wheels for collected packages: altair
Building wheel for altair (pyproject.toml) ... ?25l-
\
|
/ done
?25h Created wheel for altair: filename=altair-4.2.0.dev0-py3-none-any.whl size=812168 sha256=557424dabf704252e817b470ad2728b78049079950799f25448195b3dbb1af0d
Stored in directory: /tmp/pip-ephem-wheel-cache-0he_1o0s/wheels/06/13/e0/5bd72c969fe3954ee1561739e5c58e2ddfe5c10fcdffb12faa
Successfully built altair
Installing collected packages: toolz, altair
Successfully installed altair-4.2.0.dev0 toolz-0.11.2
Done installing Altair.
# NEEDED FOR GOOGLE COLAB
# from google.colab import auth
#from google.colab import drive
# import gspread
# from oauth2client.client import GoogleCredentials
# drive.mount('/content/drive/')
# os.chdir("/content/drive/My Drive/DCU/fouth_year/advanced_machine_learning/music-recommodation-system")
Helper functions
def calculate_sparsity(M):
"""
Computes sparsity of frequency matrix
"""
matrix_size = len((M['userID'].unique())) * len((M['artistID'].unique())) # Number of possible interactions in the matrix
num_plays = len(M['weight']) # Number of weights
sparsity = (float(num_plays/matrix_size))
return sparsity
def build_music_sparse_tensor(music_df):
"""
Args:
ratings_df: a pd.DataFrame with `userID`, `artistID` and `weight` columns.
num_rows: an integer representing the number of rows in the frequency matrix
num_rows: an integer representing the number of columns in the frequency matrix
Returns:
a tf.SparseTensor representing the feedback matrix.
"""
indices = music_df[['userID', 'artistID']].values
values = music_df['weight'].values
return tf.SparseTensor(
indices=indices,
values=values,
dense_shape=[num_users, num_artist])
def preproces_ids(music_df):
"""
Args:
ratings_df: a pd.DataFrame with `userID`, `artistID` and `weight` columns.
Returns:
a pd.DataFrame where userIDs and artistIDs now start at 1
and end at n and m (defined above), respectively
two dictionary preserving the orginal ids.
"""
unique_user_ids_list = sorted(music_df['userID'].unique())
print(unique_user_ids_list[0])
unique_user_ids = dict(zip(range(0, len(unique_user_ids_list) ),unique_user_ids_list))
unique_user_ids_switched = dict(zip(unique_user_ids_list, range(0, len(unique_user_ids) )))
unique_artist_ids_list = sorted(music_df['artistID'].unique())
unique_artist_ids = dict(zip(range(0, len(unique_artist_ids_list) ),unique_artist_ids_list))
unique_artist_ids_switched = dict(zip(unique_artist_ids_list, range(0, len(unique_artist_ids_list) )))
music_df['userID'] = music_df['userID'].map(unique_user_ids_switched)
music_df['artistID'] = music_df['artistID'].map(unique_artist_ids_switched)
return music_df, unique_user_ids, unique_artist_ids
def split_dataframe(df, holdout_fraction=0.1):
"""Splits a DataFrame into training and test sets.
Args:
df: a dataframe.
holdout_fraction: fraction of dataframe rows to use in the test set.
Returns:
train: dataframe for training
test: dataframe for testing
"""
test = df.sample(frac=holdout_fraction, replace=False)
train = df[~df.index.isin(test.index)]
return train, test
Traditional recommender system development relies on explicit feedback. Many models were designed to tackle this issue as a regression problem. For instance, the input of the model would be a matrix \(F_{nm}\) denoting user’s (m) preference of items (n) on a scale. In the classic movie ratings example, this preference would be users giving a 1-to-5 star rating to different movies.
This dataset contains implicit feedback: that is, observed logs of user interactions with items, in this instance user’s listening counts to artists. However, implicit feedback does not signal negativity, in the same way as a 1-star rating would. In our data, a user could listen to song of an artist a limited number of times. But that does not necessarily mean that the particular user has an aversion to that artist i.e. it could be part of a curated playlist by another user. Therefore, we decide to construct a binary matrix, which has a value of one if the observation is observed (i.e. a listening count has been logged between an artist and a user). Note, a 0 is not used to describe unobserved artist-user interactions. This is for optimization reasons, explained below.
user_artists = pd.read_csv('data/user_artists.dat', sep='\t')
user_artists['weight'] = 1
artists = pd.read_csv('data/artists.dat', sep='\t')
artists.rename({'id':'artistID'}, inplace=True, axis=1)
user_taggedartists = pd.read_csv(r'data/user_taggedartists-timestamps.dat', sep='\t')
user_taggedartists_years = pd.read_csv(r'data/user_taggedartists.dat', sep='\t')
tags = pd.read_csv(open('data/tags.dat', errors='replace'), sep='\t')
user_taggedartists = pd.merge(user_taggedartists, tags, on=['tagID'])
num_users = user_artists.userID.nunique()
num_artist = artists.artistID.nunique()
collab_filter_df = user_artists
Here, we calculate the top 10 tags by popularity. Then, we assign it to a artist, if the artist has a top 10 tag. If an artist’s tags are not in the top 10, we input ‘N/A’. Note, the next cell can take several mintues to compute.
top_10_tags = user_taggedartists['tagValue'].value_counts().index[0:10]
user_taggedartists['top10TagValue'] = None
for index, row in user_taggedartists.iterrows():
if row['tagValue'] in top_10_tags:
user_taggedartists.iloc[index, -1] = row['tagValue']
user_taggedartists.fillna('N/A',inplace=True)
artists = pd.merge(user_taggedartists, artists, on=['artistID'], how='right')[['artistID','name','top10TagValue','tagValue']].fillna('N/A')
artists.groupby(['artistID','name','top10TagValue']).agg(lambda x:x.value_counts().index[0]).reset_index()
artists = artists.drop_duplicates(subset=['artistID'])
assert artists.artistID.nunique() == num_artist
artists.rename({'tagValue':'mostCommonGenre'},axis=1, inplace=True)
We require two matrices or embeddings to compute a similarity measure (one for quires and one for items), but how do we get these two embeddings?
Matrix Factorisation¶
Figure 2: Data flow chart
First, we need to contsruct the feedback matrix \(F \in R^{m \times n}\), where \(m\) is the number of users and \(n\) is the number of artists. The goal is to two generate two lower-dimensional matrices \(U_{mp}\) and \(V_{np}\) ( with \(p << m\) and \(p << n\)), representing latent user and artist components, so that: $\( F \approx UV^\top \)$
First,we attempt to build the frequency matrix for both training and testing data. tf.SparseTensor is used
for efficient representation. Three separate arguments are used to represent a tensor, namely indices, values, dense_shape, where a value \(A_{ij} = a\) is encoded by setting indices[k] = [i, j] and values[k] = a. The last tensor dense_shape is used to specify the shape of the full underlying matrix. Note, as the indices arguments represent row and columns indices, some pre-processing needs to be performed on artist and user IDs. The IDs should start from 0 and end at \(m-1\) and \(n-1\) for users and artists respectively. Presently, userIDs start at 2. Two dictionaries, orginal_artist_ids, orginal_user_ids will preserve the original ids for analysis purposes later on. Assertions and print statements are used to ensure the validity of the transformations.
colab_filter_df, orginal_user_ids, orginal_artist_ids = preproces_ids(collab_filter_df)
2
colab_filter_df.describe()
| userID | artistID | weight | |
|---|---|---|---|
| count | 92834.000 | 92834.000 | 92834.000 |
| mean | 944.222 | 3235.737 | 1.000 |
| std | 546.751 | 4197.217 | 0.000 |
| min | 0.000 | 0.000 | 1.000 |
| 25% | 470.000 | 430.000 | 1.000 |
| 50% | 944.000 | 1237.000 | 1.000 |
| 75% | 1416.000 | 4266.000 | 1.000 |
| max | 1891.000 | 17631.000 | 1.000 |
Next, we caulcate the number of unique artists, userids and sparisty of our proposed frequency matrix, before splitting into training and test subsets. Quite a sparse matrix indeed!
print(f'Number of unqiue users are: {collab_filter_df["userID"].nunique()}')
print(f'Number of unqiue artists are: {collab_filter_df["artistID"].nunique()}')
print(f'Sparsity of our frequency matrix: {calculate_sparsity(collab_filter_df)}')
Number of unqiue users are: 1892
Number of unqiue artists are: 17632
Sparsity of our frequency matrix: 0.002782815119924182
collab_filter_df.to_csv('data/test_user_artists.csv',index=False)
frequency_m_train, frequency_m_test = split_dataframe(colab_filter_df)
frequency_m_train_tensor = build_music_sparse_tensor(frequency_m_train)
frequency_m_test_tensor = build_music_sparse_tensor(frequency_m_test)
assert num_users == frequency_m_train_tensor.shape.as_list()[0]
assert num_artist == frequency_m_train_tensor.shape.as_list()[1]
assert num_users == frequency_m_test_tensor.shape.as_list()[0]
assert num_artist == frequency_m_test_tensor.shape.as_list()[1]
Training a Matrix factorization model¶
Per the definition above, \(UV^\top\) approximates \(F\). The Mean Squared Error is used to measure this approximation error. In the notation below, k is used to represent the set of observed listening counts, and K is the number of observed listening counts.
However, rather than computing the full prediction matrix, \(UV^\top\) and gathering the entries in the embeddings (corresponding to the observed listening counts) , we only gather the embeddings of the observers pairs and compute their dot products. Thereby, we reduce the complexity from \(O(NM)\) to \(O(Kp)\) where \(p\) is the embedding dimension. Stochastic gradient descent (SGD) is used to minimize the loss (objective) function. The SDG algorithim cycles through the observed listening binary and caulates the prediction according to the following equation.
Then it updates the user and artist as embeddings as shown in the following equations.
where \(\alpha\) denotes the learning rate. The algorithim continues untill convergence is found.
Other matrix factorization algorithms functions are also commonly used such as Alternating Least Squares (Takács and Tikk, 2012). A modified version of the aforementioned function known as Weighted Alternating Least Squares (WALS) is slower than SDG but can be parallelised. For the purposes of this investigation, we are not particularly concerned with training times/latency requirements so we proceed with SDG.
We also decide to add regularization to our model, to avoid overfitting. Overfitting occurs when the model tries to fit the training dataset to hard and does not generalize well to unseen or future data. In the context of artist recommendation, fitting the observed listening counts often emphasizes learning high similarity (between artists with many listeners), but a good embedding representation also requires learning low similarity (between artists with few listeners).
First, we define the two classes train_matrix_norm and build_matrix_norm class. The build_matrix_norm class computes the necessary pre-processing steps before we train the model such as specifying the loss metric to optimise and the loss components( e.g. gravity loss for the regularized model) and the initial artist and user embeddings. train_matrix_norm simply trains the models and outputs figures detailing the the loss metrics and components. The methods build_vanilla() and build_reg_model() computes the necessary pre-processing steps for the non-regularized and regularized model.
### Training a Matrix Factorization model
class train_matrix_norm(object):
"""Simple class that represents a matrix normalisation model"""
def __init__(self, embedding_vars, loss, metrics=None):
"""Initializes a Matrix normalisation model
Args:
embedding_vars: A dictionary of tf.Variables.
loss: A float Tensor. The loss to optimize.
metrics: optional list of dictionaries of Tensors. The metrics in each
dictionary will be plotted in a separate figure during training.
"""
self._embedding_vars = embedding_vars
self._loss = loss
self._metrics = metrics
self._embeddings = {k: None for k in embedding_vars}
self._session = None
@property
def embeddings(self):
"""The embeddings dictionary."""
return self._embeddings
def train(self, num_iterations=100, learning_rate=1.0, plot_results=True,
optimizer=tf.train.GradientDescentOptimizer):
"""Trains the model.
Args:
iterations: number of iterations to run.
learning_rate: optimizer learning rate.
plot_results: whether to plot the results at the end of training.
optimizer: the optimizer to use. Default to SDG
Returns:
The metrics dictionary evaluated at the last iteration.
"""
with self._loss.graph.as_default():
opt = optimizer(learning_rate)
train_op = opt.minimize(self._loss)
local_init_op = tf.group(
tf.variables_initializer(opt.variables()),
tf.local_variables_initializer())
if self._session is None:
self._session = tf.Session()
with self._session.as_default():
self._session.run(tf.global_variables_initializer())
self._session.run(tf.tables_initializer())
tf.train.start_queue_runners()
with self._session.as_default():
local_init_op.run()
iterations = []
metrics = self._metrics or ({},)
metrics_vals = [collections.defaultdict(list) for _ in self._metrics]
# Train and append results.
for i in range(num_iterations + 1):
_, results = self._session.run((train_op, metrics))
if (i % 10 == 0) or i == num_iterations:
print("\r iteration %d: " % i + ", ".join(
["%s=%f" % (k, v) for r in results for k, v in r.items()]),
end='')
iterations.append(i)
for metric_val, result in zip(metrics_vals, results):
for k, v in result.items():
metric_val[k].append(v)
for k, v in self._embedding_vars.items():
self._embeddings[k] = v.eval()
if plot_results:
# Plot the metrics.
num_subplots = len(metrics)+1
fig = plt.figure()
fig.set_size_inches(num_subplots*10, 8)
for i, metric_vals in enumerate(metrics_vals):
ax = fig.add_subplot(1, num_subplots, i+1)
for k, v in metric_vals.items():
ax.plot(iterations, v, label=k)
ax.set_xlim([1, num_iterations])
ax.legend()
return results
class build_matrix_norm():
"""Simple class that represents a matrix normalisation model"""
def __init__(self, listens, embedding_dim=3, regularization_coeff=.1, gravity_coeff=1.,
init_stddev=0.1):
"""Initializes a Matrix normalisation model
Args:
listens: the DataFrame of artist listening counts.
embedding_dim: The dimension of the embedding space.
regularization_coeff: The regularization coefficient lambda.
gravity_coeff: The gravity regularization coefficient lambda_g.
Returns:
A train_matrix_norm object that uses a regularized loss.
"""
self._embedding_vars = embedding_vars
self._loss = loss
self._metrics = metrics
self._embeddings = {k: None for k in embedding_vars}
self._session = None
def sparse_mean_square_error(sparse_listens, user_embeddings, artist_embeddings):
"""
Args:
sparse_listens: A SparseTensor rating matrix, of dense_shape [N, M]
user_embeddings: A dense Tensor U of shape [N, k] where k is the embedding
dimension, such that U_i is the embedding of user i.
artist_embeddings: A dense Tensor V of shape [M, k] where k is the embedding
dimension, such that V_j is the embedding of movie j.
Returns:
A scalar Tensor representing the MSE between the true ratings and the
model's predictions.
"""
predictions = tf.gather_nd(
tf.matmul(user_embeddings, artist_embeddings, transpose_b=True),
sparse_listens.indices)
loss = tf.losses.mean_squared_error(sparse_listens.values, predictions)
return loss
def gravity(U, V):
"""Creates a gravity loss given two embedding matrices."""
return 1. / (U.shape[0].value*V.shape[0].value) * tf.reduce_sum(
tf.matmul(U, U, transpose_a=True) * tf.matmul(V, V, transpose_a=True))
def build_vanilla(embedding_dim=3, init_stddev=1.):
"""performs the necessary preprocessing steps for the regularized model. """
# Initialize the embeddings using a normal distribution.
U = tf.Variable(tf.random.normal(
[frequency_m_train_tensor.dense_shape[0], embedding_dim], stddev=init_stddev))
V = tf.Variable(tf.random.normal(
[frequency_m_train_tensor.dense_shape[1], embedding_dim], stddev=init_stddev))
embeddings = {"userID": U, "artistID": V}
error_train = build_matrix_norm.sparse_mean_square_error(frequency_m_train_tensor, U, V)
error_test = build_matrix_norm.sparse_mean_square_error(frequency_m_test_tensor, U, V)
metrics = {
'train_error': error_train,
'test_error': error_test
}
return train_matrix_norm(embeddings, error_train, [metrics])
def build_reg_model(embedding_dim=3, regularization_coeff=.1, gravity_coeff=1.,
init_stddev=0.1
):
"""performs the necessary preprocessing steps for the regularized model. """
U = tf.Variable(tf.random.normal(
[frequency_m_train_tensor.dense_shape[0], embedding_dim], stddev=init_stddev))
V = tf.Variable(tf.random.normal(
[frequency_m_train_tensor.dense_shape[1], embedding_dim], stddev=init_stddev))
embeddings = {"userID": U, "artistID": V}
error_train = build_matrix_norm.sparse_mean_square_error(frequency_m_train_tensor, U, V)
error_test = build_matrix_norm.sparse_mean_square_error(frequency_m_test_tensor, U, V)
gravity_loss = gravity_coeff * build_matrix_norm.gravity(U, V)
regularization_loss = regularization_coeff * (
tf.reduce_sum(U*U)/U.shape[0].value + tf.reduce_sum(V*V)/V.shape[0].value)
total_loss = error_train + regularization_loss + gravity_loss
losses = {
'train_error_observed': error_train,
'test_error_observed': error_test,
}
loss_components = {
'observed_loss': error_train,
'regularization_loss': regularization_loss,
'gravity_loss': gravity_loss,
}
#embeddings = {"userID": U, "artistID": V}
return train_matrix_norm(embeddings, total_loss, [losses, loss_components])
Vanilla Model (non-regularized)¶
vanilla_model = build_matrix_norm.build_vanilla(embedding_dim=35,init_stddev=.05)
vanilla_model.train(num_iterations=2000, learning_rate=20.)
2021-11-28 22:49:54.706908: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2021-11-28 22:49:54.706954: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2021-11-28 22:49:54.706980: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (fv-az42-909): /proc/driver/nvidia/version does not exist
2021-11-28 22:49:54.707279: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
iteration 0: train_error=1.000346, test_error=1.000471
iteration 10: train_error=0.998681, test_error=1.000511
iteration 20: train_error=0.996970, test_error=1.000510
iteration 30: train_error=0.995125, test_error=1.000398
iteration 40: train_error=0.993014, test_error=1.000057
iteration 50: train_error=0.990418, test_error=0.999285
iteration 60: train_error=0.986941, test_error=0.997705
iteration 70: train_error=0.981883, test_error=0.994629
iteration 80: train_error=0.974018, test_error=0.988841
iteration 90: train_error=0.961367, test_error=0.978350
iteration 100: train_error=0.941237, test_error=0.960403
iteration 110: train_error=0.911206, test_error=0.932455
iteration 120: train_error=0.871337, test_error=0.894440
iteration 130: train_error=0.825295, test_error=0.850089
iteration 140: train_error=0.777507, test_error=0.804104
iteration 150: train_error=0.730225, test_error=0.758846
iteration 160: train_error=0.684445, test_error=0.715121
iteration 170: train_error=0.641134, test_error=0.673706
iteration 180: train_error=0.601028, test_error=0.635315
iteration 190: train_error=0.564371, test_error=0.600278
iteration 200: train_error=0.531055, test_error=0.568568
iteration 210: train_error=0.500807, test_error=0.539963
iteration 220: train_error=0.473306, test_error=0.514159
iteration 230: train_error=0.448235, test_error=0.490837
iteration 240: train_error=0.425303, test_error=0.469701
iteration 250: train_error=0.404251, test_error=0.450482
iteration 260: train_error=0.384853, test_error=0.432947
iteration 270: train_error=0.366915, test_error=0.416891
iteration 280: train_error=0.350265, test_error=0.402141
iteration 290: train_error=0.334758, test_error=0.388546
iteration 300: train_error=0.320267, test_error=0.375975
iteration 310: train_error=0.306684, test_error=0.364318
iteration 320: train_error=0.293915, test_error=0.353480
iteration 330: train_error=0.281882, test_error=0.343379
iteration 340: train_error=0.270516, test_error=0.333946
iteration 350: train_error=0.259757, test_error=0.325118
iteration 360: train_error=0.249556, test_error=0.316845
iteration 370: train_error=0.239867, test_error=0.309078
iteration 380: train_error=0.230652, test_error=0.301778
iteration 390: train_error=0.221877, test_error=0.294909
iteration 400: train_error=0.213511, test_error=0.288439
iteration 410: train_error=0.205527, test_error=0.282337
iteration 420: train_error=0.197899, test_error=0.276579
iteration 430: train_error=0.190605, test_error=0.271139
iteration 440: train_error=0.183622, test_error=0.265996
iteration 450: train_error=0.176932, test_error=0.261129
iteration 460: train_error=0.170517, test_error=0.256519
iteration 470: train_error=0.164359, test_error=0.252149
iteration 480: train_error=0.158444, test_error=0.248003
iteration 490: train_error=0.152757, test_error=0.244066
iteration 500: train_error=0.147286, test_error=0.240324
iteration 510: train_error=0.142018, test_error=0.236764
iteration 520: train_error=0.136943, test_error=0.233376
iteration 530: train_error=0.132051, test_error=0.230148
iteration 540: train_error=0.127333, test_error=0.227070
iteration 550: train_error=0.122779, test_error=0.224133
iteration 560: train_error=0.118384, test_error=0.221329
iteration 570: train_error=0.114139, test_error=0.218650
iteration 580: train_error=0.110038, test_error=0.216089
iteration 590: train_error=0.106076, test_error=0.213639
iteration 600: train_error=0.102248, test_error=0.211294
iteration 610: train_error=0.098547, test_error=0.209049
iteration 620: train_error=0.094970, test_error=0.206898
iteration 630: train_error=0.091513, test_error=0.204836
iteration 640: train_error=0.088171, test_error=0.202858
iteration 650: train_error=0.084942, test_error=0.200961
iteration 660: train_error=0.081820, test_error=0.199141
iteration 670: train_error=0.078805, test_error=0.197393
iteration 680: train_error=0.075891, test_error=0.195715
iteration 690: train_error=0.073078, test_error=0.194102
iteration 700: train_error=0.070360, test_error=0.192553
iteration 710: train_error=0.067737, test_error=0.191064
iteration 720: train_error=0.065206, test_error=0.189632
iteration 730: train_error=0.062763, test_error=0.188254
iteration 740: train_error=0.060407, test_error=0.186930
iteration 750: train_error=0.058136, test_error=0.185655
iteration 760: train_error=0.055946, test_error=0.184428
iteration 770: train_error=0.053836, test_error=0.183248
iteration 780: train_error=0.051804, test_error=0.182111
iteration 790: train_error=0.049846, test_error=0.181016
iteration 800: train_error=0.047962, test_error=0.179961
iteration 810: train_error=0.046149, test_error=0.178945
iteration 820: train_error=0.044404, test_error=0.177966
iteration 830: train_error=0.042726, test_error=0.177023
iteration 840: train_error=0.041112, test_error=0.176114
iteration 850: train_error=0.039560, test_error=0.175237
iteration 860: train_error=0.038069, test_error=0.174392
iteration 870: train_error=0.036636, test_error=0.173576
iteration 880: train_error=0.035259, test_error=0.172790
iteration 890: train_error=0.033936, test_error=0.172031
iteration 900: train_error=0.032666, test_error=0.171299
iteration 910: train_error=0.031446, test_error=0.170593
iteration 920: train_error=0.030274, test_error=0.169911
iteration 930: train_error=0.029150, test_error=0.169253
iteration 940: train_error=0.028070, test_error=0.168617
iteration 950: train_error=0.027033, test_error=0.168004
iteration 960: train_error=0.026038, test_error=0.167411
iteration 970: train_error=0.025083, test_error=0.166838
iteration 980: train_error=0.024165, test_error=0.166285
iteration 990: train_error=0.023285, test_error=0.165750
iteration 1000: train_error=0.022440, test_error=0.165234
iteration 1010: train_error=0.021628, test_error=0.164734
iteration 1020: train_error=0.020849, test_error=0.164251
iteration 1030: train_error=0.020101, test_error=0.163784
iteration 1040: train_error=0.019383, test_error=0.163333
iteration 1050: train_error=0.018693, test_error=0.162896
iteration 1060: train_error=0.018030, test_error=0.162473
iteration 1070: train_error=0.017394, test_error=0.162065
iteration 1080: train_error=0.016783, test_error=0.161669
iteration 1090: train_error=0.016195, test_error=0.161286
iteration 1100: train_error=0.015631, test_error=0.160916
iteration 1110: train_error=0.015089, test_error=0.160557
iteration 1120: train_error=0.014568, test_error=0.160209
iteration 1130: train_error=0.014067, test_error=0.159873
iteration 1140: train_error=0.013585, test_error=0.159547
iteration 1150: train_error=0.013122, test_error=0.159231
iteration 1160: train_error=0.012677, test_error=0.158925
iteration 1170: train_error=0.012249, test_error=0.158629
iteration 1180: train_error=0.011837, test_error=0.158342
iteration 1190: train_error=0.011441, test_error=0.158063
iteration 1200: train_error=0.011060, test_error=0.157793
iteration 1210: train_error=0.010694, test_error=0.157532
iteration 1220: train_error=0.010341, test_error=0.157278
iteration 1230: train_error=0.010001, test_error=0.157032
iteration 1240: train_error=0.009675, test_error=0.156793
iteration 1250: train_error=0.009360, test_error=0.156561
iteration 1260: train_error=0.009057, test_error=0.156336
iteration 1270: train_error=0.008766, test_error=0.156118
iteration 1280: train_error=0.008485, test_error=0.155906
iteration 1290: train_error=0.008214, test_error=0.155701
iteration 1300: train_error=0.007954, test_error=0.155501
iteration 1310: train_error=0.007703, test_error=0.155307
iteration 1320: train_error=0.007461, test_error=0.155119
iteration 1330: train_error=0.007228, test_error=0.154936
iteration 1340: train_error=0.007004, test_error=0.154758
iteration 1350: train_error=0.006787, test_error=0.154586
iteration 1360: train_error=0.006579, test_error=0.154418
iteration 1370: train_error=0.006378, test_error=0.154255
iteration 1380: train_error=0.006184, test_error=0.154096
iteration 1390: train_error=0.005997, test_error=0.153942
iteration 1400: train_error=0.005817, test_error=0.153792
iteration 1410: train_error=0.005643, test_error=0.153647
iteration 1420: train_error=0.005475, test_error=0.153505
iteration 1430: train_error=0.005314, test_error=0.153367
iteration 1440: train_error=0.005158, test_error=0.153233
iteration 1450: train_error=0.005007, test_error=0.153103
iteration 1460: train_error=0.004862, test_error=0.152976
iteration 1470: train_error=0.004722, test_error=0.152852
iteration 1480: train_error=0.004587, test_error=0.152732
iteration 1490: train_error=0.004456, test_error=0.152615
iteration 1500: train_error=0.004330, test_error=0.152500
iteration 1510: train_error=0.004208, test_error=0.152389
iteration 1520: train_error=0.004091, test_error=0.152281
iteration 1530: train_error=0.003977, test_error=0.152176
iteration 1540: train_error=0.003868, test_error=0.152073
iteration 1550: train_error=0.003762, test_error=0.151973
iteration 1560: train_error=0.003660, test_error=0.151875
iteration 1570: train_error=0.003561, test_error=0.151780
iteration 1580: train_error=0.003465, test_error=0.151688
iteration 1590: train_error=0.003373, test_error=0.151597
iteration 1600: train_error=0.003284, test_error=0.151509
iteration 1610: train_error=0.003198, test_error=0.151423
iteration 1620: train_error=0.003114, test_error=0.151339
iteration 1630: train_error=0.003034, test_error=0.151258
iteration 1640: train_error=0.002956, test_error=0.151178
iteration 1650: train_error=0.002881, test_error=0.151100
iteration 1660: train_error=0.002808, test_error=0.151024
iteration 1670: train_error=0.002738, test_error=0.150950
iteration 1680: train_error=0.002669, test_error=0.150877
iteration 1690: train_error=0.002603, test_error=0.150807
iteration 1700: train_error=0.002540, test_error=0.150738
iteration 1710: train_error=0.002478, test_error=0.150670
iteration 1720: train_error=0.002418, test_error=0.150605
iteration 1730: train_error=0.002360, test_error=0.150540
iteration 1740: train_error=0.002304, test_error=0.150477
iteration 1750: train_error=0.002250, test_error=0.150416
iteration 1760: train_error=0.002198, test_error=0.150356
iteration 1770: train_error=0.002147, test_error=0.150297
iteration 1780: train_error=0.002098, test_error=0.150240
iteration 1790: train_error=0.002050, test_error=0.150184
iteration 1800: train_error=0.002004, test_error=0.150129
iteration 1810: train_error=0.001959, test_error=0.150075
iteration 1820: train_error=0.001916, test_error=0.150023
iteration 1830: train_error=0.001874, test_error=0.149971
iteration 1840: train_error=0.001833, test_error=0.149921
iteration 1850: train_error=0.001794, test_error=0.149872
iteration 1860: train_error=0.001755, test_error=0.149824
iteration 1870: train_error=0.001718, test_error=0.149777
iteration 1880: train_error=0.001682, test_error=0.149731
iteration 1890: train_error=0.001647, test_error=0.149686
iteration 1900: train_error=0.001613, test_error=0.149642
iteration 1910: train_error=0.001580, test_error=0.149598
iteration 1920: train_error=0.001548, test_error=0.149556
iteration 1930: train_error=0.001517, test_error=0.149515
iteration 1940: train_error=0.001487, test_error=0.149474
iteration 1950: train_error=0.001458, test_error=0.149434
iteration 1960: train_error=0.001430, test_error=0.149395
iteration 1970: train_error=0.001402, test_error=0.149357
iteration 1980: train_error=0.001376, test_error=0.149319
iteration 1990: train_error=0.001350, test_error=0.149283
iteration 2000: train_error=0.001324, test_error=0.149247
[{'train_error': 0.0013244184, 'test_error': 0.14924659}]
Regularized moodel¶
reg_model = build_matrix_norm.build_reg_model(regularization_coeff=0.1, gravity_coeff=1.0, embedding_dim=35,init_stddev=.05)
reg_model.train(num_iterations=2000, learning_rate=20.)
iteration 0: train_error_observed=1.000341, test_error_observed=1.000450, observed_loss=1.000341, regularization_loss=0.017547, gravity_loss=0.000220
iteration 10: train_error_observed=0.998708, test_error_observed=1.000440, observed_loss=0.998708, regularization_loss=0.017138, gravity_loss=0.000210
iteration 20: train_error_observed=0.997115, test_error_observed=1.000402, observed_loss=0.997115, regularization_loss=0.016781, gravity_loss=0.000201
iteration 30: train_error_observed=0.995478, test_error_observed=1.000271, observed_loss=0.995478, regularization_loss=0.016476, gravity_loss=0.000194
iteration 40: train_error_observed=0.993684, test_error_observed=0.999949, observed_loss=0.993684, regularization_loss=0.016221, gravity_loss=0.000187
iteration 50: train_error_observed=0.991543, test_error_observed=0.999265, observed_loss=0.991543, regularization_loss=0.016021, gravity_loss=0.000183
iteration 60: train_error_observed=0.988732, test_error_observed=0.997913, observed_loss=0.988732, regularization_loss=0.015883, gravity_loss=0.000179
iteration 70: train_error_observed=0.984679, test_error_observed=0.995332, observed_loss=0.984679, regularization_loss=0.015827, gravity_loss=0.000178
iteration 80: train_error_observed=0.978399, test_error_observed=0.990545, observed_loss=0.978399, regularization_loss=0.015886, gravity_loss=0.000180
iteration 90: train_error_observed=0.968306, test_error_observed=0.981954, observed_loss=0.968306, regularization_loss=0.016118, gravity_loss=0.000188
iteration 100: train_error_observed=0.952207, test_error_observed=0.967309, observed_loss=0.952207, regularization_loss=0.016615, gravity_loss=0.000207
iteration 110: train_error_observed=0.927957, test_error_observed=0.944350, observed_loss=0.927957, regularization_loss=0.017495, gravity_loss=0.000249
iteration 120: train_error_observed=0.895110, test_error_observed=0.912505, observed_loss=0.895110, regularization_loss=0.018862, gravity_loss=0.000336
iteration 130: train_error_observed=0.856167, test_error_observed=0.874324, observed_loss=0.856167, regularization_loss=0.020725, gravity_loss=0.000497
iteration 140: train_error_observed=0.814862, test_error_observed=0.833868, observed_loss=0.814862, regularization_loss=0.022981, gravity_loss=0.000757
iteration 150: train_error_observed=0.773321, test_error_observed=0.793552, observed_loss=0.773321, regularization_loss=0.025496, gravity_loss=0.001128
iteration 160: train_error_observed=0.732218, test_error_observed=0.753984, observed_loss=0.732218, regularization_loss=0.028183, gravity_loss=0.001623
iteration 170: train_error_observed=0.692250, test_error_observed=0.715606, observed_loss=0.692250, regularization_loss=0.030989, gravity_loss=0.002251
iteration 180: train_error_observed=0.654308, test_error_observed=0.679136, observed_loss=0.654308, regularization_loss=0.033860, gravity_loss=0.003019
iteration 190: train_error_observed=0.619022, test_error_observed=0.645178, observed_loss=0.619022, regularization_loss=0.036733, gravity_loss=0.003920
iteration 200: train_error_observed=0.586633, test_error_observed=0.614022, observed_loss=0.586633, regularization_loss=0.039557, gravity_loss=0.004944
iteration 210: train_error_observed=0.557111, test_error_observed=0.585692, observed_loss=0.557111, regularization_loss=0.042291, gravity_loss=0.006071
iteration 220: train_error_observed=0.530292, test_error_observed=0.560059, observed_loss=0.530292, regularization_loss=0.044913, gravity_loss=0.007284
iteration 230: train_error_observed=0.505953, test_error_observed=0.536917, observed_loss=0.505953, regularization_loss=0.047410, gravity_loss=0.008567
iteration 240: train_error_observed=0.483851, test_error_observed=0.516027, observed_loss=0.483851, regularization_loss=0.049774, gravity_loss=0.009904
iteration 250: train_error_observed=0.463752, test_error_observed=0.497151, observed_loss=0.463752, regularization_loss=0.052007, gravity_loss=0.011280
iteration 260: train_error_observed=0.445435, test_error_observed=0.480066, observed_loss=0.445435, regularization_loss=0.054108, gravity_loss=0.012683
iteration 270: train_error_observed=0.428703, test_error_observed=0.464569, observed_loss=0.428703, regularization_loss=0.056084, gravity_loss=0.014102
iteration 280: train_error_observed=0.413380, test_error_observed=0.450478, observed_loss=0.413380, regularization_loss=0.057939, gravity_loss=0.015528
iteration 290: train_error_observed=0.399312, test_error_observed=0.437634, observed_loss=0.399312, regularization_loss=0.059678, gravity_loss=0.016953
iteration 300: train_error_observed=0.386360, test_error_observed=0.425899, observed_loss=0.386360, regularization_loss=0.061310, gravity_loss=0.018370
iteration 310: train_error_observed=0.374406, test_error_observed=0.415150, observed_loss=0.374406, regularization_loss=0.062839, gravity_loss=0.019773
iteration 320: train_error_observed=0.363343, test_error_observed=0.405280, observed_loss=0.363343, regularization_loss=0.064272, gravity_loss=0.021159
iteration 330: train_error_observed=0.353079, test_error_observed=0.396195, observed_loss=0.353079, regularization_loss=0.065616, gravity_loss=0.022521
iteration 340: train_error_observed=0.343533, test_error_observed=0.387815, observed_loss=0.343533, regularization_loss=0.066876, gravity_loss=0.023859
iteration 350: train_error_observed=0.334636, test_error_observed=0.380067, observed_loss=0.334636, regularization_loss=0.068059, gravity_loss=0.025168
iteration 360: train_error_observed=0.326324, test_error_observed=0.372890, observed_loss=0.326324, regularization_loss=0.069169, gravity_loss=0.026447
iteration 370: train_error_observed=0.318542, test_error_observed=0.366228, observed_loss=0.318542, regularization_loss=0.070212, gravity_loss=0.027694
iteration 380: train_error_observed=0.311244, test_error_observed=0.360034, observed_loss=0.311244, regularization_loss=0.071192, gravity_loss=0.028908
iteration 390: train_error_observed=0.304387, test_error_observed=0.354266, observed_loss=0.304387, regularization_loss=0.072115, gravity_loss=0.030089
iteration 400: train_error_observed=0.297934, test_error_observed=0.348885, observed_loss=0.297934, regularization_loss=0.072983, gravity_loss=0.031234
iteration 410: train_error_observed=0.291850, test_error_observed=0.343860, observed_loss=0.291850, regularization_loss=0.073801, gravity_loss=0.032345
iteration 420: train_error_observed=0.286106, test_error_observed=0.339159, observed_loss=0.286106, regularization_loss=0.074573, gravity_loss=0.033420
iteration 430: train_error_observed=0.280676, test_error_observed=0.334758, observed_loss=0.280676, regularization_loss=0.075302, gravity_loss=0.034459
iteration 440: train_error_observed=0.275536, test_error_observed=0.330631, observed_loss=0.275536, regularization_loss=0.075990, gravity_loss=0.035463
iteration 450: train_error_observed=0.270662, test_error_observed=0.326758, observed_loss=0.270662, regularization_loss=0.076641, gravity_loss=0.036432
iteration 460: train_error_observed=0.266037, test_error_observed=0.323119, observed_loss=0.266037, regularization_loss=0.077258, gravity_loss=0.037365
iteration 470: train_error_observed=0.261641, test_error_observed=0.319697, observed_loss=0.261641, regularization_loss=0.077842, gravity_loss=0.038264
iteration 480: train_error_observed=0.257459, test_error_observed=0.316476, observed_loss=0.257459, regularization_loss=0.078396, gravity_loss=0.039129
iteration 490: train_error_observed=0.253474, test_error_observed=0.313440, observed_loss=0.253474, regularization_loss=0.078923, gravity_loss=0.039960
iteration 500: train_error_observed=0.249674, test_error_observed=0.310576, observed_loss=0.249674, regularization_loss=0.079424, gravity_loss=0.040757
iteration 510: train_error_observed=0.246045, test_error_observed=0.307873, observed_loss=0.246045, regularization_loss=0.079902, gravity_loss=0.041522
iteration 520: train_error_observed=0.242575, test_error_observed=0.305319, observed_loss=0.242575, regularization_loss=0.080358, gravity_loss=0.042256
iteration 530: train_error_observed=0.239254, test_error_observed=0.302903, observed_loss=0.239254, regularization_loss=0.080793, gravity_loss=0.042958
iteration 540: train_error_observed=0.236071, test_error_observed=0.300616, observed_loss=0.236071, regularization_loss=0.081210, gravity_loss=0.043630
iteration 550: train_error_observed=0.233018, test_error_observed=0.298449, observed_loss=0.233018, regularization_loss=0.081610, gravity_loss=0.044273
iteration 560: train_error_observed=0.230085, test_error_observed=0.296394, observed_loss=0.230085, regularization_loss=0.081995, gravity_loss=0.044886
iteration 570: train_error_observed=0.227266, test_error_observed=0.294444, observed_loss=0.227266, regularization_loss=0.082365, gravity_loss=0.045473
iteration 580: train_error_observed=0.224551, test_error_observed=0.292591, observed_loss=0.224551, regularization_loss=0.082722, gravity_loss=0.046032
iteration 590: train_error_observed=0.221935, test_error_observed=0.290830, observed_loss=0.221935, regularization_loss=0.083067, gravity_loss=0.046565
iteration 600: train_error_observed=0.219411, test_error_observed=0.289153, observed_loss=0.219411, regularization_loss=0.083401, gravity_loss=0.047072
iteration 610: train_error_observed=0.216973, test_error_observed=0.287557, observed_loss=0.216973, regularization_loss=0.083726, gravity_loss=0.047556
iteration 620: train_error_observed=0.214616, test_error_observed=0.286035, observed_loss=0.214616, regularization_loss=0.084041, gravity_loss=0.048015
iteration 630: train_error_observed=0.212335, test_error_observed=0.284584, observed_loss=0.212335, regularization_loss=0.084349, gravity_loss=0.048452
iteration 640: train_error_observed=0.210125, test_error_observed=0.283198, observed_loss=0.210125, regularization_loss=0.084649, gravity_loss=0.048867
iteration 650: train_error_observed=0.207981, test_error_observed=0.281873, observed_loss=0.207981, regularization_loss=0.084943, gravity_loss=0.049261
iteration 660: train_error_observed=0.205901, test_error_observed=0.280607, observed_loss=0.205901, regularization_loss=0.085231, gravity_loss=0.049634
iteration 670: train_error_observed=0.203879, test_error_observed=0.279394, observed_loss=0.203879, regularization_loss=0.085515, gravity_loss=0.049988
iteration 680: train_error_observed=0.201912, test_error_observed=0.278233, observed_loss=0.201912, regularization_loss=0.085794, gravity_loss=0.050322
iteration 690: train_error_observed=0.199998, test_error_observed=0.277120, observed_loss=0.199998, regularization_loss=0.086069, gravity_loss=0.050639
iteration 700: train_error_observed=0.198132, test_error_observed=0.276053, observed_loss=0.198132, regularization_loss=0.086340, gravity_loss=0.050938
iteration 710: train_error_observed=0.196313, test_error_observed=0.275028, observed_loss=0.196313, regularization_loss=0.086609, gravity_loss=0.051220
iteration 720: train_error_observed=0.194538, test_error_observed=0.274044, observed_loss=0.194538, regularization_loss=0.086876, gravity_loss=0.051486
iteration 730: train_error_observed=0.192804, test_error_observed=0.273097, observed_loss=0.192804, regularization_loss=0.087140, gravity_loss=0.051737
iteration 740: train_error_observed=0.191108, test_error_observed=0.272187, observed_loss=0.191108, regularization_loss=0.087403, gravity_loss=0.051972
iteration 750: train_error_observed=0.189450, test_error_observed=0.271310, observed_loss=0.189450, regularization_loss=0.087664, gravity_loss=0.052193
iteration 760: train_error_observed=0.187827, test_error_observed=0.270466, observed_loss=0.187827, regularization_loss=0.087925, gravity_loss=0.052401
iteration 770: train_error_observed=0.186237, test_error_observed=0.269652, observed_loss=0.186237, regularization_loss=0.088185, gravity_loss=0.052595
iteration 780: train_error_observed=0.184678, test_error_observed=0.268867, observed_loss=0.184678, regularization_loss=0.088444, gravity_loss=0.052777
iteration 790: train_error_observed=0.183149, test_error_observed=0.268109, observed_loss=0.183149, regularization_loss=0.088703, gravity_loss=0.052946
iteration 800: train_error_observed=0.181648, test_error_observed=0.267378, observed_loss=0.181648, regularization_loss=0.088962, gravity_loss=0.053104
iteration 810: train_error_observed=0.180175, test_error_observed=0.266671, observed_loss=0.180175, regularization_loss=0.089221, gravity_loss=0.053250
iteration 820: train_error_observed=0.178727, test_error_observed=0.265987, observed_loss=0.178727, regularization_loss=0.089480, gravity_loss=0.053386
iteration 830: train_error_observed=0.177304, test_error_observed=0.265326, observed_loss=0.177304, regularization_loss=0.089740, gravity_loss=0.053511
iteration 840: train_error_observed=0.175904, test_error_observed=0.264687, observed_loss=0.175904, regularization_loss=0.090000, gravity_loss=0.053627
iteration 850: train_error_observed=0.174527, test_error_observed=0.264067, observed_loss=0.174527, regularization_loss=0.090260, gravity_loss=0.053733
iteration 860: train_error_observed=0.173172, test_error_observed=0.263467, observed_loss=0.173172, regularization_loss=0.090522, gravity_loss=0.053830
iteration 870: train_error_observed=0.171837, test_error_observed=0.262885, observed_loss=0.171837, regularization_loss=0.090784, gravity_loss=0.053918
iteration 880: train_error_observed=0.170523, test_error_observed=0.262321, observed_loss=0.170523, regularization_loss=0.091047, gravity_loss=0.053998
iteration 890: train_error_observed=0.169227, test_error_observed=0.261774, observed_loss=0.169227, regularization_loss=0.091311, gravity_loss=0.054069
iteration 900: train_error_observed=0.167950, test_error_observed=0.261243, observed_loss=0.167950, regularization_loss=0.091576, gravity_loss=0.054134
iteration 910: train_error_observed=0.166691, test_error_observed=0.260727, observed_loss=0.166691, regularization_loss=0.091841, gravity_loss=0.054190
iteration 920: train_error_observed=0.165449, test_error_observed=0.260226, observed_loss=0.165449, regularization_loss=0.092108, gravity_loss=0.054240
iteration 930: train_error_observed=0.164224, test_error_observed=0.259739, observed_loss=0.164224, regularization_loss=0.092375, gravity_loss=0.054283
iteration 940: train_error_observed=0.163015, test_error_observed=0.259266, observed_loss=0.163015, regularization_loss=0.092643, gravity_loss=0.054320
iteration 950: train_error_observed=0.161822, test_error_observed=0.258806, observed_loss=0.161822, regularization_loss=0.092912, gravity_loss=0.054350
iteration 960: train_error_observed=0.160644, test_error_observed=0.258358, observed_loss=0.160644, regularization_loss=0.093182, gravity_loss=0.054375
iteration 970: train_error_observed=0.159482, test_error_observed=0.257923, observed_loss=0.159482, regularization_loss=0.093453, gravity_loss=0.054394
iteration 980: train_error_observed=0.158334, test_error_observed=0.257499, observed_loss=0.158334, regularization_loss=0.093724, gravity_loss=0.054408
iteration 990: train_error_observed=0.157200, test_error_observed=0.257086, observed_loss=0.157200, regularization_loss=0.093997, gravity_loss=0.054416
iteration 1000: train_error_observed=0.156080, test_error_observed=0.256684, observed_loss=0.156080, regularization_loss=0.094269, gravity_loss=0.054419
iteration 1010: train_error_observed=0.154973, test_error_observed=0.256293, observed_loss=0.154973, regularization_loss=0.094543, gravity_loss=0.054418
iteration 1020: train_error_observed=0.153880, test_error_observed=0.255911, observed_loss=0.153880, regularization_loss=0.094817, gravity_loss=0.054412
iteration 1030: train_error_observed=0.152801, test_error_observed=0.255539, observed_loss=0.152801, regularization_loss=0.095091, gravity_loss=0.054402
iteration 1040: train_error_observed=0.151734, test_error_observed=0.255177, observed_loss=0.151734, regularization_loss=0.095366, gravity_loss=0.054388
iteration 1050: train_error_observed=0.150679, test_error_observed=0.254823, observed_loss=0.150679, regularization_loss=0.095641, gravity_loss=0.054370
iteration 1060: train_error_observed=0.149638, test_error_observed=0.254478, observed_loss=0.149638, regularization_loss=0.095916, gravity_loss=0.054349
iteration 1070: train_error_observed=0.148608, test_error_observed=0.254142, observed_loss=0.148608, regularization_loss=0.096191, gravity_loss=0.054323
iteration 1080: train_error_observed=0.147591, test_error_observed=0.253813, observed_loss=0.147591, regularization_loss=0.096467, gravity_loss=0.054295
iteration 1090: train_error_observed=0.146585, test_error_observed=0.253493, observed_loss=0.146585, regularization_loss=0.096743, gravity_loss=0.054263
iteration 1100: train_error_observed=0.145592, test_error_observed=0.253180, observed_loss=0.145592, regularization_loss=0.097018, gravity_loss=0.054228
iteration 1110: train_error_observed=0.144610, test_error_observed=0.252875, observed_loss=0.144610, regularization_loss=0.097293, gravity_loss=0.054190
iteration 1120: train_error_observed=0.143640, test_error_observed=0.252577, observed_loss=0.143640, regularization_loss=0.097569, gravity_loss=0.054149
iteration 1130: train_error_observed=0.142681, test_error_observed=0.252286, observed_loss=0.142681, regularization_loss=0.097843, gravity_loss=0.054106
iteration 1140: train_error_observed=0.141733, test_error_observed=0.252001, observed_loss=0.141733, regularization_loss=0.098118, gravity_loss=0.054060
iteration 1150: train_error_observed=0.140796, test_error_observed=0.251723, observed_loss=0.140796, regularization_loss=0.098392, gravity_loss=0.054012
iteration 1160: train_error_observed=0.139871, test_error_observed=0.251452, observed_loss=0.139871, regularization_loss=0.098665, gravity_loss=0.053962
iteration 1170: train_error_observed=0.138956, test_error_observed=0.251187, observed_loss=0.138956, regularization_loss=0.098938, gravity_loss=0.053909
iteration 1180: train_error_observed=0.138052, test_error_observed=0.250927, observed_loss=0.138052, regularization_loss=0.099210, gravity_loss=0.053855
iteration 1190: train_error_observed=0.137159, test_error_observed=0.250674, observed_loss=0.137159, regularization_loss=0.099482, gravity_loss=0.053798
iteration 1200: train_error_observed=0.136277, test_error_observed=0.250426, observed_loss=0.136277, regularization_loss=0.099753, gravity_loss=0.053740
iteration 1210: train_error_observed=0.135405, test_error_observed=0.250184, observed_loss=0.135405, regularization_loss=0.100022, gravity_loss=0.053680
iteration 1220: train_error_observed=0.134543, test_error_observed=0.249947, observed_loss=0.134543, regularization_loss=0.100291, gravity_loss=0.053618
iteration 1230: train_error_observed=0.133692, test_error_observed=0.249716, observed_loss=0.133692, regularization_loss=0.100559, gravity_loss=0.053555
iteration 1240: train_error_observed=0.132851, test_error_observed=0.249490, observed_loss=0.132851, regularization_loss=0.100825, gravity_loss=0.053491
iteration 1250: train_error_observed=0.132019, test_error_observed=0.249268, observed_loss=0.132019, regularization_loss=0.101091, gravity_loss=0.053425
iteration 1260: train_error_observed=0.131198, test_error_observed=0.249052, observed_loss=0.131198, regularization_loss=0.101355, gravity_loss=0.053357
iteration 1270: train_error_observed=0.130387, test_error_observed=0.248840, observed_loss=0.130387, regularization_loss=0.101618, gravity_loss=0.053289
iteration 1280: train_error_observed=0.129586, test_error_observed=0.248633, observed_loss=0.129586, regularization_loss=0.101880, gravity_loss=0.053219
iteration 1290: train_error_observed=0.128794, test_error_observed=0.248430, observed_loss=0.128794, regularization_loss=0.102140, gravity_loss=0.053149
iteration 1300: train_error_observed=0.128012, test_error_observed=0.248231, observed_loss=0.128012, regularization_loss=0.102399, gravity_loss=0.053077
iteration 1310: train_error_observed=0.127239, test_error_observed=0.248037, observed_loss=0.127239, regularization_loss=0.102657, gravity_loss=0.053005
iteration 1320: train_error_observed=0.126476, test_error_observed=0.247847, observed_loss=0.126476, regularization_loss=0.102913, gravity_loss=0.052932
iteration 1330: train_error_observed=0.125721, test_error_observed=0.247662, observed_loss=0.125721, regularization_loss=0.103167, gravity_loss=0.052858
iteration 1340: train_error_observed=0.124977, test_error_observed=0.247480, observed_loss=0.124977, regularization_loss=0.103420, gravity_loss=0.052783
iteration 1350: train_error_observed=0.124241, test_error_observed=0.247302, observed_loss=0.124241, regularization_loss=0.103671, gravity_loss=0.052707
iteration 1360: train_error_observed=0.123514, test_error_observed=0.247127, observed_loss=0.123514, regularization_loss=0.103921, gravity_loss=0.052631
iteration 1370: train_error_observed=0.122796, test_error_observed=0.246957, observed_loss=0.122796, regularization_loss=0.104169, gravity_loss=0.052554
iteration 1380: train_error_observed=0.122086, test_error_observed=0.246790, observed_loss=0.122086, regularization_loss=0.104415, gravity_loss=0.052477
iteration 1390: train_error_observed=0.121386, test_error_observed=0.246626, observed_loss=0.121386, regularization_loss=0.104660, gravity_loss=0.052400
iteration 1400: train_error_observed=0.120694, test_error_observed=0.246466, observed_loss=0.120694, regularization_loss=0.104903, gravity_loss=0.052321
iteration 1410: train_error_observed=0.120010, test_error_observed=0.246309, observed_loss=0.120010, regularization_loss=0.105144, gravity_loss=0.052243
iteration 1420: train_error_observed=0.119335, test_error_observed=0.246156, observed_loss=0.119335, regularization_loss=0.105383, gravity_loss=0.052164
iteration 1430: train_error_observed=0.118667, test_error_observed=0.246006, observed_loss=0.118667, regularization_loss=0.105621, gravity_loss=0.052085
iteration 1440: train_error_observed=0.118008, test_error_observed=0.245858, observed_loss=0.118008, regularization_loss=0.105856, gravity_loss=0.052005
iteration 1450: train_error_observed=0.117357, test_error_observed=0.245714, observed_loss=0.117357, regularization_loss=0.106090, gravity_loss=0.051926
iteration 1460: train_error_observed=0.116714, test_error_observed=0.245573, observed_loss=0.116714, regularization_loss=0.106322, gravity_loss=0.051846
iteration 1470: train_error_observed=0.116079, test_error_observed=0.245435, observed_loss=0.116079, regularization_loss=0.106553, gravity_loss=0.051766
iteration 1480: train_error_observed=0.115451, test_error_observed=0.245300, observed_loss=0.115451, regularization_loss=0.106781, gravity_loss=0.051686
iteration 1490: train_error_observed=0.114831, test_error_observed=0.245167, observed_loss=0.114831, regularization_loss=0.107008, gravity_loss=0.051605
iteration 1500: train_error_observed=0.114219, test_error_observed=0.245037, observed_loss=0.114219, regularization_loss=0.107232, gravity_loss=0.051525
iteration 1510: train_error_observed=0.113614, test_error_observed=0.244910, observed_loss=0.113614, regularization_loss=0.107455, gravity_loss=0.051444
iteration 1520: train_error_observed=0.113016, test_error_observed=0.244785, observed_loss=0.113016, regularization_loss=0.107676, gravity_loss=0.051364
iteration 1530: train_error_observed=0.112425, test_error_observed=0.244663, observed_loss=0.112425, regularization_loss=0.107896, gravity_loss=0.051283
iteration 1540: train_error_observed=0.111842, test_error_observed=0.244543, observed_loss=0.111842, regularization_loss=0.108113, gravity_loss=0.051203
iteration 1550: train_error_observed=0.111265, test_error_observed=0.244426, observed_loss=0.111265, regularization_loss=0.108328, gravity_loss=0.051122
iteration 1560: train_error_observed=0.110695, test_error_observed=0.244311, observed_loss=0.110695, regularization_loss=0.108542, gravity_loss=0.051042
iteration 1570: train_error_observed=0.110132, test_error_observed=0.244199, observed_loss=0.110132, regularization_loss=0.108754, gravity_loss=0.050962
iteration 1580: train_error_observed=0.109576, test_error_observed=0.244088, observed_loss=0.109576, regularization_loss=0.108964, gravity_loss=0.050881
iteration 1590: train_error_observed=0.109027, test_error_observed=0.243980, observed_loss=0.109027, regularization_loss=0.109172, gravity_loss=0.050801
iteration 1600: train_error_observed=0.108483, test_error_observed=0.243875, observed_loss=0.108483, regularization_loss=0.109379, gravity_loss=0.050721
iteration 1610: train_error_observed=0.107947, test_error_observed=0.243771, observed_loss=0.107947, regularization_loss=0.109583, gravity_loss=0.050641
iteration 1620: train_error_observed=0.107416, test_error_observed=0.243669, observed_loss=0.107416, regularization_loss=0.109786, gravity_loss=0.050562
iteration 1630: train_error_observed=0.106892, test_error_observed=0.243570, observed_loss=0.106892, regularization_loss=0.109987, gravity_loss=0.050482
iteration 1640: train_error_observed=0.106374, test_error_observed=0.243472, observed_loss=0.106374, regularization_loss=0.110186, gravity_loss=0.050403
iteration 1650: train_error_observed=0.105862, test_error_observed=0.243376, observed_loss=0.105862, regularization_loss=0.110384, gravity_loss=0.050324
iteration 1660: train_error_observed=0.105356, test_error_observed=0.243283, observed_loss=0.105356, regularization_loss=0.110580, gravity_loss=0.050245
iteration 1670: train_error_observed=0.104856, test_error_observed=0.243191, observed_loss=0.104856, regularization_loss=0.110774, gravity_loss=0.050166
iteration 1680: train_error_observed=0.104362, test_error_observed=0.243101, observed_loss=0.104362, regularization_loss=0.110966, gravity_loss=0.050087
iteration 1690: train_error_observed=0.103873, test_error_observed=0.243013, observed_loss=0.103873, regularization_loss=0.111157, gravity_loss=0.050009
iteration 1700: train_error_observed=0.103390, test_error_observed=0.242926, observed_loss=0.103390, regularization_loss=0.111346, gravity_loss=0.049931
iteration 1710: train_error_observed=0.102913, test_error_observed=0.242841, observed_loss=0.102913, regularization_loss=0.111534, gravity_loss=0.049853
iteration 1720: train_error_observed=0.102441, test_error_observed=0.242758, observed_loss=0.102441, regularization_loss=0.111719, gravity_loss=0.049776
iteration 1730: train_error_observed=0.101974, test_error_observed=0.242677, observed_loss=0.101974, regularization_loss=0.111903, gravity_loss=0.049699
iteration 1740: train_error_observed=0.101513, test_error_observed=0.242597, observed_loss=0.101513, regularization_loss=0.112086, gravity_loss=0.049622
iteration 1750: train_error_observed=0.101057, test_error_observed=0.242519, observed_loss=0.101057, regularization_loss=0.112267, gravity_loss=0.049545
iteration 1760: train_error_observed=0.100606, test_error_observed=0.242443, observed_loss=0.100606, regularization_loss=0.112446, gravity_loss=0.049469
iteration 1770: train_error_observed=0.100160, test_error_observed=0.242367, observed_loss=0.100160, regularization_loss=0.112624, gravity_loss=0.049393
iteration 1780: train_error_observed=0.099720, test_error_observed=0.242294, observed_loss=0.099720, regularization_loss=0.112800, gravity_loss=0.049317
iteration 1790: train_error_observed=0.099284, test_error_observed=0.242222, observed_loss=0.099284, regularization_loss=0.112975, gravity_loss=0.049242
iteration 1800: train_error_observed=0.098853, test_error_observed=0.242151, observed_loss=0.098853, regularization_loss=0.113148, gravity_loss=0.049167
iteration 1810: train_error_observed=0.098427, test_error_observed=0.242082, observed_loss=0.098427, regularization_loss=0.113319, gravity_loss=0.049092
iteration 1820: train_error_observed=0.098005, test_error_observed=0.242014, observed_loss=0.098005, regularization_loss=0.113489, gravity_loss=0.049018
iteration 1830: train_error_observed=0.097588, test_error_observed=0.241947, observed_loss=0.097588, regularization_loss=0.113658, gravity_loss=0.048944
iteration 1840: train_error_observed=0.097176, test_error_observed=0.241882, observed_loss=0.097176, regularization_loss=0.113825, gravity_loss=0.048870
iteration 1850: train_error_observed=0.096768, test_error_observed=0.241818, observed_loss=0.096768, regularization_loss=0.113991, gravity_loss=0.048797
iteration 1860: train_error_observed=0.096365, test_error_observed=0.241756, observed_loss=0.096365, regularization_loss=0.114155, gravity_loss=0.048724
iteration 1870: train_error_observed=0.095966, test_error_observed=0.241695, observed_loss=0.095966, regularization_loss=0.114318, gravity_loss=0.048652
iteration 1880: train_error_observed=0.095572, test_error_observed=0.241635, observed_loss=0.095572, regularization_loss=0.114480, gravity_loss=0.048579
iteration 1890: train_error_observed=0.095182, test_error_observed=0.241576, observed_loss=0.095182, regularization_loss=0.114640, gravity_loss=0.048507
iteration 1900: train_error_observed=0.094796, test_error_observed=0.241518, observed_loss=0.094796, regularization_loss=0.114799, gravity_loss=0.048436
iteration 1910: train_error_observed=0.094414, test_error_observed=0.241462, observed_loss=0.094414, regularization_loss=0.114956, gravity_loss=0.048365
iteration 1920: train_error_observed=0.094036, test_error_observed=0.241406, observed_loss=0.094036, regularization_loss=0.115112, gravity_loss=0.048294
iteration 1930: train_error_observed=0.093663, test_error_observed=0.241352, observed_loss=0.093663, regularization_loss=0.115267, gravity_loss=0.048223
iteration 1940: train_error_observed=0.093293, test_error_observed=0.241299, observed_loss=0.093293, regularization_loss=0.115420, gravity_loss=0.048153
iteration 1950: train_error_observed=0.092927, test_error_observed=0.241247, observed_loss=0.092927, regularization_loss=0.115572, gravity_loss=0.048084
iteration 1960: train_error_observed=0.092565, test_error_observed=0.241196, observed_loss=0.092565, regularization_loss=0.115723, gravity_loss=0.048014
iteration 1970: train_error_observed=0.092207, test_error_observed=0.241146, observed_loss=0.092207, regularization_loss=0.115873, gravity_loss=0.047945
iteration 1980: train_error_observed=0.091853, test_error_observed=0.241097, observed_loss=0.091853, regularization_loss=0.116021, gravity_loss=0.047877
iteration 1990: train_error_observed=0.091502, test_error_observed=0.241049, observed_loss=0.091502, regularization_loss=0.116168, gravity_loss=0.047808
iteration 2000: train_error_observed=0.091155, test_error_observed=0.241002, observed_loss=0.091155, regularization_loss=0.116314, gravity_loss=0.047740
[{'train_error_observed': 0.09115525, 'test_error_observed': 0.24100238},
{'observed_loss': 0.09115525,
'regularization_loss': 0.11631441,
'gravity_loss': 0.047740445}]
In both models, we observe a steep loss in train error and test as the model progress. Although, the regularized model has a higher MSE, both on the training and test set. It must be noted that the quality of recommendation is improved when regularization is added, which is proven when the artist_neighbors() function is utilized. In addition, we observe in the end evaluation section, that the the performance of the model is improved when regularization is added. The test error decreases similarity to the test error, although it plateaus around the 1000 epoch mark. As expected, the the additional loss generated by the regularization functions increases over epochs. We add the following regularisation terms to our model.
Regularization of the model parameters. This is a common \(\ell_2\) regularization term on the embedding matrices, given by \(r(U, V) = \frac{1}{N} \sum_i \|U_i\|^2 + \frac{1}{M}\sum_j \|V_j\|^2\).
A global prior that pushes the prediction of any pair towards zero, called the gravity term. This is given by \(g(U, V) = \frac{1}{MN} \sum_{i = 1}^N \sum_{j = 1}^M \langle U_i, V_j \rangle^2\)
These terms modifies the “global” loss (as in, the sum of the network loss and the regularization loss) in order to drive the optimization algorithm in desired directions i.e. prevent overfitting.
Evaluating the embeddings¶
We will use two similairty meausres to inspect the robustness of our system:
Dot product: score of artist j \(\langle u, V_j \rangle\).
Cosine angle: score of artist j \(\frac{\langle u, V_j \rangle}{\|u\|\|V_j\|}\).
DOT = 'dot'
COSINE = 'cosine'
def compute_scores(query_embedding, item_embeddings, measure=DOT):
"""Computes the scores of the candidates given a query.
Args:
query_embedding: a vector of shape [k], representing the query embedding.
item_embeddings: a matrix of shape [N, k], such that row i is the embedding
of item i.
measure: a string specifying the similarity measure to be used. Can be
either DOT or COSINE.
Returns:
scores: a vector of shape [N], such that scores[i] is the score of item i.
"""
u = query_embedding
V = item_embeddings
if measure == COSINE:
V = V / np.linalg.norm(V, axis=1, keepdims=True)
u = u / np.linalg.norm(u)
scores = u.dot(V.T)
return scores
def user_recommendations(model,user_id, k=15, measure=DOT, exclude_rated=False):
scores = compute_scores(
model.embeddings["userID"][user_id], model.embeddings["artistID"], measure)
score_key = measure + ' score'
df = pd.DataFrame({
'score': list(scores),
'name': artists.sort_values('artistID', ascending=True)['name'],
'most assigned tag':artists.sort_values('artistID', ascending=True)['mostCommonGenre']
})
return df.sort_values(['score'], ascending=False).head(k)
def artist_neighbors(model, title_substring, measure=DOT, k=6):
# Search for artist ids that match the given substring.
inv_artist_id_mapping = {v: k for k, v in orginal_artist_ids.items()}
ids = artists[artists['name'].str.contains(title_substring)].artistID.values
titles = artists[artists.artistID.isin(ids)]['name'].values
if len(titles) == 0:
raise ValueError("Found no artists with name %s" % title_substring)
print("Nearest neighbors of : %s." % titles[0])
if len(titles) > 1:
print("[Found more than one matching artist. Other candidates: {}]".format(
", ".join(titles[1:])))
artists_id_orginal = ids[0]
asrtists_id_mapped = inv_artist_id_mapping[ids[0]]
scores = compute_scores(
model.embeddings["artistID"][asrtists_id_mapped], model.embeddings["artistID"],
measure)
score_key = measure + ' score'
df = pd.DataFrame({
score_key: list(scores),
'name': artists.sort_values('artistID', ascending=True)['name'],
'most assigned tag':artists.sort_values('artistID', ascending=True)['mostCommonGenre']
})
return df.sort_values([score_key], ascending=False).head(k)
Here, we find the most similar artists to the band the cure. We also include the most assigned tag associated with an artist. The reccomdations are conistent with our domain knowedge of bands similar to the cure.
artist_neighbors(vanilla_model, "The Cure", DOT)
Nearest neighbors of : The Cure.
| dot score | name | most assigned tag | |
|---|---|---|---|
| 9437 | 0.546 | The Cure | chillout |
| 84188 | 0.531 | Pato Fu | pop |
| 57843 | 0.531 | Nightwish | pop |
| 71605 | 0.530 | Guns N' Roses | atmospheric |
| 17278 | 0.530 | Kings of Leon | chillout |
| 58990 | 0.530 | Queen | 80's |
artist_neighbors(vanilla_model, "The Cure", COSINE)
Nearest neighbors of : The Cure.
| cosine score | name | most assigned tag | |
|---|---|---|---|
| 9437 | 1.000 | The Cure | chillout |
| 71605 | 0.959 | Guns N' Roses | atmospheric |
| 8273 | 0.957 | Radiohead | chillout |
| 58990 | 0.953 | Queen | 80's |
| 15847 | 0.951 | Red Hot Chili Peppers | chillout |
| 10850 | 0.951 | Placebo | chillout |
artist_neighbors(reg_model, "The Cure", DOT)
Nearest neighbors of : The Cure.
| dot score | name | most assigned tag | |
|---|---|---|---|
| 16680 | 3.261 | The Beatles | chillout |
| 9437 | 3.185 | The Cure | chillout |
| 12363 | 3.176 | Muse | chillout |
| 3259 | 3.166 | Coldplay | chillout |
| 18364 | 3.163 | Nirvana | pop |
| 15847 | 3.147 | Red Hot Chili Peppers | chillout |
artist_neighbors(reg_model, "The Cure", COSINE)
Nearest neighbors of : The Cure.
| cosine score | name | most assigned tag | |
|---|---|---|---|
| 9437 | 1.000 | The Cure | chillout |
| 43413 | 0.963 | David Bowie | chillout |
| 8273 | 0.958 | Radiohead | chillout |
| 32942 | 0.958 | The Smiths | groove |
| 38968 | 0.958 | U2 | electronic |
| 4936 | 0.952 | Depeche Mode | chillout |
We observe that dot product tends to recommends more popular artists such as Nirvana and The Beatles, where as Cosine Similarity recommends more obscure artists. This is likely due to the fact that the norm of the embedding in matrix factorization is often correlated with prevalence. The regularised model seems to output better reccomodations as the varation of the most assigned tag attribute is less when compared to the vanilla model. In addition, Marilyn Manson was recommended by the vanilla model in our intial run. We argue that these artists are most dis-similar! However, this observation is subject to change when you run the model, as we initialize the embedddings with a random gaussian generator.
def artist_embedding_norm(models):
"""Visualizes the norm and number of ratings of the artist embeddings.
Args:
model: A train_matrix_norm object.
"""
if not isinstance(models, list):
models = [models]
df = pd.DataFrame({
'name': artists.sort_values('artistID', ascending=True)['name'].values,
'number of user-artist interactions': user_artists[['artistID','userID']].sort_values('artistID', ascending=True).groupby('artistID').count()['userID'].values,
})
charts = []
brush = alt.selection_interval()
for i, model in enumerate(models):
norm_key = 'norm'+str(i)
df[norm_key] = np.linalg.norm(model.embeddings["artistID"], axis=1)
nearest = alt.selection(
type='single', encodings=['x', 'y'], on='mouseover', nearest=True,
empty='none')
base = alt.Chart().mark_circle().encode(
x='number of user-artist interactions',
y=norm_key,
color=alt.condition(brush, alt.value('#4c78a8'), alt.value('lightgray'))
).properties(
selection=nearest).add_selection(brush)
text = alt.Chart().mark_text(align='center', dx=5, dy=-5).encode(
x='number of user-artist interactions', y=norm_key,
text=alt.condition(nearest, 'name', alt.value('')))
charts.append(alt.layer(base, text))
return alt.hconcat(*charts, data=df)
artist_embedding_norm(reg_model)
def visualize_movie_embeddings(data, x, y):
genre_filter = alt.selection_multi(fields=['top10TagValue'])
genre_chart = alt.Chart().mark_bar().encode(
x="count()",
y=alt.Y('top10TagValue'),
color=alt.condition(
genre_filter,
alt.Color("top10TagValue:N"),
alt.value('lightgray'))
).properties(height=300, selection=genre_filter)
nearest = alt.selection(
type='single', encodings=['x', 'y'], on='mouseover', nearest=True,
empty='none')
base = alt.Chart().mark_circle().encode(
x=x,
y=y,
color=alt.condition(genre_filter, "top10TagValue", alt.value("whitesmoke")),
).properties(
width=600,
height=600,
selection=nearest)
text = alt.Chart().mark_text(align='left', dx=5, dy=-5).encode(
x=x,
y=y,
text=alt.condition(nearest, 'name', alt.value('')))
return alt.hconcat(alt.layer(base, text), genre_chart, data=data)
def tsne_movie_embeddings(model):
"""Visualizes the movie embeddings, projected using t-SNE with Cosine measure.
Args:
model: A MFModel object.
"""
tsne = sklearn.manifold.TSNE(
n_components=2, perplexity=40, metric='cosine', early_exaggeration=10.0,
init='pca', verbose=True, n_iter=400)
print('Running t-SNE...')
V_proj = tsne.fit_transform(model.embeddings["artistID"])
artists.loc[:,'x'] = V_proj[:, 0]
artists.loc[:,'y'] = V_proj[:, 1]
return visualize_movie_embeddings(artists, 'x', 'y')
T-distributed stochastic neighbor embedding (t-SNE) is a dimensionality reduction algorithm useful for visualizing high dimensional data. We use this algorithim to visualise our embeddings of the regualrised model. Due to the large number of user submitted semantic categories, we decide to color-code the top 15 tags, with the rest being labelled as ‘N/A’. Although the sea of orange, indicating’N/A’, makes it difficult to interrupt these results, the regularised model seems to adequaltly cluster artists of a similar genre in it’s embeddings.
tsne_movie_embeddings(reg_model)
Running t-SNE...
[t-SNE] Computing 121 nearest neighbors...
[t-SNE] Indexed 17632 samples in 0.001s...
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/sklearn/manifold/_t_sne.py:793: FutureWarning: The default learning rate in TSNE will change from 200.0 to 'auto' in 1.2.
FutureWarning,
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/sklearn/manifold/_t_sne.py:827: FutureWarning: 'square_distances' has been introduced in 0.24 to help phase out legacy squaring behavior. The 'legacy' setting will be removed in 1.1 (renaming of 0.26), and the default setting will be changed to True. In 1.3, 'square_distances' will be removed altogether, and distances will be squared by default. Set 'square_distances'=True to silence this warning.
FutureWarning,
[t-SNE] Computed neighbors for 17632 samples in 5.322s...
[t-SNE] Computed conditional probabilities for sample 1000 / 17632
[t-SNE] Computed conditional probabilities for sample 2000 / 17632
[t-SNE] Computed conditional probabilities for sample 3000 / 17632
[t-SNE] Computed conditional probabilities for sample 4000 / 17632
[t-SNE] Computed conditional probabilities for sample 5000 / 17632
[t-SNE] Computed conditional probabilities for sample 6000 / 17632
[t-SNE] Computed conditional probabilities for sample 7000 / 17632
[t-SNE] Computed conditional probabilities for sample 8000 / 17632
[t-SNE] Computed conditional probabilities for sample 9000 / 17632
[t-SNE] Computed conditional probabilities for sample 10000 / 17632
[t-SNE] Computed conditional probabilities for sample 11000 / 17632
[t-SNE] Computed conditional probabilities for sample 12000 / 17632
[t-SNE] Computed conditional probabilities for sample 13000 / 17632
[t-SNE] Computed conditional probabilities for sample 14000 / 17632
[t-SNE] Computed conditional probabilities for sample 15000 / 17632
[t-SNE] Computed conditional probabilities for sample 16000 / 17632
[t-SNE] Computed conditional probabilities for sample 17000 / 17632
[t-SNE] Computed conditional probabilities for sample 17632 / 17632
[t-SNE] Mean sigma: 0.179662
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/sklearn/manifold/_t_sne.py:986: FutureWarning: The PCA initialization in TSNE will change to have the standard deviation of PC1 equal to 1e-4 in 1.2. This will ensure better convergence.
FutureWarning,
[t-SNE] KL divergence after 250 iterations with early exaggeration: 77.142731
[t-SNE] KL divergence after 400 iterations: 2.762864
def m_embedding_norm(models):
"""Visualizes the norm and number of ratings of the movie embeddings.
Args:
model: A MFModel object.
"""
if not isinstance(models, list):
models = [models]
df = pd.DataFrame({
'title': artists.sort_values('artistID', ascending=True)['name'].values,
'num_ratings': user_artists[['artistID','userID']].sort_values('artistID', ascending=True).groupby('artistID').count()['userID'].values,
})
charts = []
brush = alt.selection_interval()
for i, model in enumerate(models):
norm_key = 'norm'+str(i)
df[norm_key] = np.linalg.norm(model.embeddings["artistID"], axis=1)
nearest = alt.selection(
type='single', encodings=['x', 'y'], on='mouseover', nearest=True,
empty='none')
base = alt.Chart().mark_circle().encode(
x='num_ratings',
y=norm_key,
color=alt.condition(brush, alt.value('#4c78a8'), alt.value('lightgray'))
).properties(
selection=nearest).add_selection(brush)
text = alt.Chart().mark_text(align='center', dx=5, dy=-5).encode(
x='num_ratings', y=norm_key,
text=alt.condition(nearest, 'title', alt.value('')))
charts.append(alt.layer(base, text))
return alt.hconcat(*charts, data=df)
Demo¶
You can find the most similar artist to a specified artist (that is contained in Last.FM) using the artist_neighbours() function. Similarily, you can find the top 10 recommendations of a particular userID [0 to 1891] using the user_recommendations() function. The first argument specifies the desired model, second argument the userID and third the top-k recommendations. Fourth argument represents the similarity measure, either DOT or COSINE (default = DOT, not a string).
user_recommendations(reg_model, 234, 10, COSINE)
| score | name | most assigned tag | |
|---|---|---|---|
| 126400 | 0.933 | The Vibrators | punk |
| 126582 | 0.914 | Validuaté | N/A |
| 126554 | 0.912 | Moreira da Silva | N/A |
| 126491 | 0.902 | Bandas Gaúchas - www.DownsMtv.com | N/A |
| 126513 | 0.897 | Graforréia Xilarmônica | rock |
| 126539 | 0.892 | Menstruação Anarquika | N/A |
| 126451 | 0.880 | Tim Maia | pop |
| 126583 | 0.848 | The Saints | punk |
| 126490 | 0.833 | Street Bulldogs | N/A |
| 126536 | 0.827 | S.O.D. | thrash metal |
To further demonstrate the robustness of the system and measure the serendipity of our model, we incorporate the top artists that we listen to on Spotify (i.e. an unknown user). Note, these artists have to also be in the Last.FM dataset. The recommendation system should output similar artists based on it’s artist embeddings. The Spotipy library is used to interact with Spotify’s API. The similarity measure used is the Dot product. Due to the short lived nature of the spotify token and the fact you have to sign into a pop-up to retrieve the authentication token, we simply list our top 5 artists manually. If we did not, jupyter book will stall when attempting to build as it is waiting for our response. However, we provide the code used to retrieve the short-lived token for verification purposes.
"""
import spotipy
from spotipy.oauth2 import SpotifyOAuth
client_id = <insert_your_client_id>
client_secret = <insert your client secret>
redirect_url = '<insert your redirect uri>
scope = "user-top-read user-read-playback-state streaming ugc-image-upload playlist-modify-public"
authenticate_manager = spotipy.oauth2.SpotifyOAuth(client_id = client_id,client_secret = client_secret,redirect_uri =redirect_url,scope =scope,show_dialog = True)
sp = spotipy.Spotify(auth_manager=authenticate_manager)
artists_long = sp.current_user_top_artists(limit=5, time_range="long_term")
"""
top_5_artists =[
'Coldplay',
'Paramore',
'Arctic Monkeys',
'Lily Allen',
'Miley Cyrus'
]
spotify_reccomdations_df = pd.DataFrame()
for artist in top_5_artists:
similar_artist_df = artist_neighbors(reg_model, artist)[['name','dot score']]
spotify_reccomdations_df = pd.concat([spotify_reccomdations_df, similar_artist_df])
spotify_reccomdations_df.sort_values('dot score', ascending=False).head(10)
Nearest neighbors of : Coldplay.
[Found more than one matching artist. Other candidates: Jay-Z & Coldplay, Coldplay/U2]
Nearest neighbors of : Paramore.
[Found more than one matching artist. Other candidates: Paramore攀]
Nearest neighbors of : Arctic Monkeys.
[Found more than one matching artist. Other candidates: Arctic Monkeys vs The Killers]
Nearest neighbors of : Lily Allen.
Nearest neighbors of : Miley Cyrus.
[Found more than one matching artist. Other candidates: Miley Cyrus攀, Demi Lovato Ft. Miley Cyrus Ft. Selena Gomez Ft. Jonas Brothers, Miley Cyrus and Billy Ray Cyrus, Miley Cyrus and John Travolta, Hannah Montana and Miley Cyrus]
| name | dot score | |
|---|---|---|
| 3259 | Coldplay | 3.670 |
| 12363 | Muse | 3.595 |
| 37842 | Paramore | 3.537 |
| 17472 | The Killers | 3.531 |
| 24447 | Lily Allen | 3.486 |
| 6543 | Lady Gaga | 3.477 |
| 6543 | Lady Gaga | 3.476 |
| 6543 | Lady Gaga | 3.474 |
| 17832 | Green Day | 3.466 |
| 16680 | The Beatles | 3.464 |
We believe these recommodations are good as when our model was given an artist in the top five, it actually recommended other artits in the top five.
Evaluation Code¶
This is the code needed to produce the in-depth model comparison. As we decided to use different notebooks for different models, the results of this code will be combined and explained later in the book.
## create holdout test set for each user (15 items)
user_artists = pd.read_csv('data/user_artists.dat', sep='\t')
user_ids = []
holdout_artits = []
for user_id in user_artists.userID.unique():
top_15_artists = user_artists[user_artists.userID == user_id].sort_values(by='weight').head(15).artistID.tolist()
if len(top_15_artists) == 15:
holdout_artits.append(top_15_artists)
user_ids.append(user_id)
holdout_df = pd.DataFrame(data={'userID':user_ids,'holdout_artists':holdout_artits})
holdout_df.to_csv('data/evaluation/test-set.csv',index=False)
## Finding the models vanilla, regualrised predection for each user.
def get_top_15_model_predictions(model, measure):
"""Computes the top 15 predictions for a given model
Args:
model: the name of the model
measure: a string specifying the similarity measure to be used. Can be
either DOT or COSINE.
Returns:
predicted_df a dataframe containing userIDs, their top 15 artists by the model, and the correspnding scores.
"""
artist_name_id_dict = dict(zip(artists['name'], artists['artistID']))
user_ids = []
predicted_artists = []
scores_list = []
for new_user_id, orginal_user_id in orginal_user_ids.items():
top_15_names = user_recommendations(model, new_user_id, k=15,measure=measure )['name'].values
top_15_scores = user_recommendations(model, new_user_id, k=15, measure=measure )['score'].values.tolist()
artist_ids = []
for name in top_15_names:
artist_ids.append(artist_name_id_dict[name])
predicted_artists.append(artist_ids)
user_ids.append(orginal_user_id)
scores_list.append(top_15_scores)
predicted_df = pd.DataFrame(data={'userID':user_ids,'predictions_artists':predicted_artists, 'score':scores_list })
return predicted_df
# save the recommended artits into dfs and save them to data/evaluation folder
vanilla_dot_pred= get_top_15_model_predictions(vanilla_model, measure=DOT)
vanilla_cos_pred = get_top_15_model_predictions(vanilla_model, measure=COSINE)
reg_dot_pred= get_top_15_model_predictions(reg_model, measure=DOT)
reg_cos_pred = get_top_15_model_predictions(reg_model, measure=COSINE)
vanilla_dot_pred.to_csv('data/evaluation/vannila_dot_pred.csv',index=False)
vanilla_cos_pred.to_csv('data/evaluation/vanila_cos_pred.csv',index=False)
reg_dot_pred.to_csv('data/evaluation/reg_dot_pred.csv',index=False)
reg_cos_pred.to_csv('data/evaluation/reg_cos_pred.csv',index=False)